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ABSTRACT 

This paper discusses evaluating 
Limited~English~prof icient (LEP) teacher training and inservice 
programs. It is suggested that this kind of teacher training can 
provide teachers with assistance necessary to increase the academic 
performance of linguistically and culturally diverse students. The 
history of teacher education evaluation, particularly its 
methodology, is reviewed, and current recommendations from research 
on effective education of linguistically diverse students are 
examined^ Experiences with an evaluated preservice and an inservice 
teacher education program are reported. The preservice program was a 
University of Hawaii alternative program and the inservice program 
was part of the California New Teicher Project at the University of 
California at Santa Cruz. Based on the presentation of teacher 
education program evaluation literature, the findings of recent 
research on effective teaching and learning models for linguistic 
minorities and the experiences of the preservice and inservice 
programs, the paper concludes with recommendations for LEP preservice 
and inservice teacher education program evaluation. (VWL) 
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Evaluating Limited English Proficient (LEP) 
Teacher Training 
and In-Service Programs 

Stephanie Dalton 
National Center for the Study of Cultural Diversity 
and Second Language Learning 

and 

Ellen Moir 
Board of Studies in Education 
University of California, Santa Cruz 

The topic of this paper, Evaluating LEP Toacher Training and 
In-Service Programs, is, unfortunately, among the least reported is- 
sues in the literature of teacher education research. The conse- 
quences of this neglect are evident not only in program evaluation's 
underdevelopment and in unexamined teacher education progiams 
but also in the indi^/idual experiences of increasing numbers of teach- 
ers nationwide. LEP Teacher training and in-service programs can 
provide teachers with the assistance necessary to increase the aca- 
demic performance of linguistically and culturally diverse students. 
There is a growing knowledge base regarding the content and peda- 
gogy of effective education of LEP students. In addition, qualitative 
and quantitative methodologies ofFer diverse measures for obtaining 
useful data about program experience to apply in program develop- 
ment. 

In this paper, the authors first briefly review the history of 
teacher education evaluation, particularly its methodology, and then 
examine content recommendations coming from current research on 
effective education of linguistically diverse students. Secondly, they 
report their experiences with an evaluated preservice and an in-ser- 
vice teacher education program. The preservice progi'am was a Uni- 
versity of Hawaii alternative program titled Preservice Education for 
Teachers of Jlinorities (PETOM) and the in-service program was 
part of the California New Teacher Project (CNTP) at the University 
of California at Santa Cruz. Based on the presentation of teacher 
education program evaluation literature, the findings of recent re- 
search on effective teaching and learning models for linguistic mi- 
norities, and the experiences of the preservice and in-service pro- 
grams, the paper will conclude with recommendations for LEP 
preservice and in-service teacher education program evaluation. The 
recommendations place emphasis on generating evaluation designs 
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that yield useful information for teacher education program develop- 
ers and faculty, the use of broad based methods to obtain multiple 
perspectives on program experience and effect, and the desirabihty of 
program evaluation design that informs and involves all partici- 
pants. 



Introduction 

A literature search in 1988 for documentation of multicultural 
program evaluation produced a single reference. This situation has 
changed Uttle since for programs addressing educational issues of 
cultural or linguistic diversity. To underscore the void in this do- 
main, teacher education program evaluation, in general, has been 
referred to as "teacher education's orphan" (Galluzzo and Craig, 
1990). 

Cooper's (1983) summary of program evaluation in teacher edu- 
cation included a very short list of institutions (such as the Univer- 
sity of Georgia and Ohio State University) where evaluation was an 
integral part of teacher education program development. Although 
few examples exist in the Uterature, survey responses indicate that 
more evaluation is actually practiced. In the same year as Cooper's 
summary, Adams & Craig (1983) conducted a survey of institutions 
affiliated with the American Association of Colleges for Teacher Edu- 
cation (AACTE). The survey, a follow-up questionnaire, was mailed 
to teacher education programs across the nation. The survey identi- 
fied four hundred institutions that gathered program evaluation 
data 

These reports suggest a possible mismatch of program evaluation 
literature and practice. It appears that program evaluation is indeed 
conducted but primarily for accountability reports to accrediting or 
other external agencies. Certainly program evaluation design must 
meet the reporting requirements and criteria of official agencies such 
as the National Council for Accreditation of Teacher Education 
(NCATE), State Commissions for Teacher CredentiaHng and the Na- 
tional Association of State Directors for Teacher Education Certifica- 
tion (NASDTEC). 

It appears that in practice little program evaluation is specifi- 
cally designed for internal use in program improvement or to in- 
crease understanding about developmental processes. This means 
that the suitability of teacher education curricula for the communi- 
ties served, the effect of program on professional and LEP student 
consumers, and experiences of progi'am participants remain largely 
unexplored. With so few examples of evaluated programs available, 
teacher education programs experience little pressure to evaluate. 
Evaluation's low priority in preservice and in-service teacher educa- 
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tion program development is one explanation for programs' unre- 
sponsiveness to rapidly changing teaching conditions. 

First Recommendation 

Our first recommendation is that LEP teacher education and in- 
service programs employing evaluation need to be identified and 
documented. More examples will pro\dde the models, explore the 
process, and stimulate the installation of an evaluation component in 
programs. 

We know the nation is facing a major challenge in reshaping its 
schools to be appropriate to the diversity of its population. The pub- 
lic school demographic trends are changing for teachers and stu- 
dents. For example, the state of California has positions for 11,000 
bilingual teachers. Last year, the University of California trained 
272 bilingual teachers. An OBEMLA sponsored forum on "Staffing 
the Multilingually Impacted Schools of the 1990s" produced evidence 
from the participating administrators indicating a current shortage 
of 175,000 bilingual teachers nationwide if a 20:1 student- teacher ra- 
tio is considered. Demographic imbalance between non-minority 
teachers and minority students means that teachers will be working 
with students whose backgrounds are culturally and linguistically 
different from their own. Sometimes, as in the case in the Los Ange- 
les School District, veteran teachers are finding the demographics of 
their classroom changing from year to year and from familiar cul- 
tural and linguistic backgrounds to those that are totally unfamiliar. 

LEP teacher training and in-service programs can provide teach- 
ers with the assistance necessary to increase the academic perfor- 
mance of linguistically and culturally diverse students. ^^Tiile there 
is yet no template for effective LEP student instruction, there is a 
substantial knowledge base regarding the content and pedagogy of 
effective education of LEP students. For example, the National Cen- 
ter for Research on Cultural Diversity and Second Language Learn- 
ing is currently engaged in identifying and documenting the instruc- 
tional practices of effective teachers of culturally and linguistically 
diverse minorities. These findings will provide the working models 
of effective instructional practice informing the continuing debate 
regarding the education of linguistic minorities. 

Teacher education programs can use these effective models for 
educating LEP students as resources for their own program develop- 
ment. Programs desiring to import information from models, the 
knowledge base on effective education for linguistic and culturally 
diverse students, new research findings, and other sources design 
evaluation to aid this process. The University of Hawaii and Califor- 
nia programs described in this paper have program evaluation com- 
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ponents. The notion of evaluation design emerging from these pro- 
gram experiences is one that obtains useful feedback about program 
operations as well as processes. Importantly, in this notion is the 
means that programs use to respond to evaluative feedback and dis- 
cover their own developmental processes. 

Second Recommendation 

The second recommendation is directed to the issue of program 
responsiveness. Program responsiveness to evaluation feedback 
needs documentation. Evaluation methodology designed to yield rel- 
evant, substantive, and useful information for program developers 
and faculty is most likely to result in program responsiveness to feed- 
back. More examples from evaluated programs will encourage this 
practice and demonstrate useful evaluation design. 

The history of the documentation of teacher education evalua- 
tion, particularly its methodology, is worth reviewing for models of 
effective and less effective LEP program evaluation documentation. 
Content recommendations coming from current research on effective 
education of linguistically diverse students relate to evaluation is- 
sues and influence program evaluation form and construction. The 
trends and themes in the history of teacher education program 
evaluation can inform LEP teacher education program evaluation. 

Although there is no evidence that linguistically diverse students 
were ever norm group members or that models influencing evalua- 
tion design were validated on them, the record of experience is useful 
for understanding what methods produce useful information for pro- 
gram developers. The earliest evaluation reports dating from the 
1940s relied on models of evaluation emphasizing goal attainment, 
product orientation, and teacher performance competency based on 
program objectives. 

A 1944 publication by Troyer and Pace described methods used 
by teacher education institutions to evaluate programs. They identi- 
fied program components such as the general education component, 
the professional education sequence, student teaching and follow-up 
studies and described a variety of methods for assessing the skills, 
attitudes, and understandings of preservice teachers. According to 
Galluzzo and Craig (1990), teacher 'education evaluation has only 
minimally shifted since the 1944 documentation. As in an AACTE 
survey by Adams & Craig (1983), program evaluation reports con- 
tinue to rely heavily on single tap follow-up surveys. 

The preponderance of follow-up studies that consist of post- 
graduates' self-reports raise issues of reliability and validity. The 
usefulness of such data for program development has been rightly 
challenged. For example, Katz et al. (1981) critically reviewed 26 
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studies using graduates' responses to follow-up questionnaires as 
evaluation data. Here Katz et al. contributed the "feed-forward"* 
principle. Feed-forward is defined as the "resistance from the stu- 
dent at the time of exposure to given learnings and, later, protesta- 
tions that the same learnings had not been provided, should have 
been provided or should have been provided in stronger doses" (p. 
21). This situation illustrates the substantial validity limitations on 
data gathered through questionnaires from individuals after the ex- 
perience of the program. Katz et al. call for evaluation data that will 
be more informative for program effectiveness and development. 

In 1970, Sandefur's monograph on a model for program evalua- 
tion described a product-oriented, competency-based approach. In 
this outcomes based mode!, selected competencies served as objec- 
tives of the program for student and new teacher performance. This 
model has been re-energized in the 1987 National Council for Ac- 
creditation of Teacher Education (NCATE) guidelines which empha- 
size the degree to which students achieve objectives of a program. 
Medley (1977) in addressing teacher education program evaluation 
defined it as the extent to which "the training experiences produce 
the competencies defined as objectives of the training program" 
(Medley, 1977, p.69). 

In the 1980s, the value of alternative approaches including in- 
creased description and documentation of on-going program experi- 
ence from the participants' point of view were recognized and called 
for. Qualitative methodology was recommended for capturing pro- 
gram features previously ignored. These features included program 
antecedents, contexts of program operation, intended audience, de- 
velopmental and process issues. This approach meant previously un- 
explored issues relating to student and teacher language and think- 
ing development could be studied. 

In sum, design and implementation of on-going teacher education 
program evaluation is currently defined in wider ranging terms than 
the narrowly conceived outcomes-based orientation that has been 
predominant. Methodology that has been overladen with self-report 
particularly from graduate surveys has produced data that program 
developers are hard pressed to find a use for. The link betv/een ac- 
tual program practices and teacher development, as a result, is sub- 
stantially unexamined. 

Third Recommendation 

Our third recommendation for program evaluation supports 
methodological approaches employing quantitative and qualitative 
inquiry methods for the purpose of exploring processes and transfor- 
mations produced by progi'ams and experienced by preservice and in- 
service teachers. The program itself is engaging a process in design- 



419 ^ 



ing and conducting evaluation and this is worthy of attention. The 
use of broad based methods and measures that obtain multiple per- 
spectives on program experience and effect is most likely to capture 
information about developmental processes and the links between 
program and sources of influence. 

Issues ofLEP Student Education 

In reviewing the recommendations coming from current research 
about augmenting linguistic minority student achievement, consen- 
sus on every issue is rare, but understanding of the issues in the 
form of a substantial knowledge base is accumulating. New data are 
providing better understanding of the cognitive assets associated 
with high degrees of bilingualism (Diaz, 1983, Hakuta, 1986), Lan- 
guage learning research is indicating that second language develop- 
ment is not significantly impeded by native language and human 
cognition is indeed organized to accommodate new language learning 
(McLaughlin, 1990). The position that bilingualism is a deficit condi- 
tion is no longer sustainable. 

Among these themes surrounding the education of LEP students, 
one of the most central concerns is the instructional use of the lan- 
guages of bilinguals in classrooms (Garcia, 1990). Recent research 
reports of Wong-Fillmore, Ramirez and the growing influence of 
VygDtskian theory inform our understanding about the critical role 
of language in the education of LEP students, 

Wong-Fillmore's (1991) position that children of linguistic mi- 
norities are assimilating into English at the expense of their home 
language is another theme of the debate. Capitalizing on the com- 
mon belief that the younger a learner is the faster and more com- 
pletely a new language can be learned, states have legislated 
younger and younger English-only instruction. What has been ig- 
nored in this policy is the cost to the young learners and their fami- 
lies in primary language loss - what has been referred to as "sub- 
tractive bilingualism" (Wong-Fillmore, p. 1, 1991). New language 
learning is not dependent on inattention to native language and the 
social consequences of this approach to LEP student education have 
been made sadly clear. 

Ramirez' (1991) longitudinal study compared the relative effec- 
tiveness of three bilingual programs: (1) structured English immer- 
sion strategy, (2) late-exit and (3) early-exit transitional bilingual 
education. The study*s findings strongly support the effectiveness of 
bilingual programs and indicate that students exposed to more En- 
glish in English Immersion programs perform no better overall on 
tests of English langUvige ability than do students in early- or late- 
exit bilingual programs. 
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Of particular interest for this topic, Ramirez found teachers' re- 
sponses to the challenge of heterogeneous LEP classrooms to be re- 
markably similar in that a minimum of language production opportu- 
nities were provided for the students. When students were sepa- 
rated by language classifications of LEP, Fluent English Speakers 
(FEP) and English Only (EO), there was no difference in teachers' 
talk among the three programs. In mixed classes of LEP, FEP and 
EOs, teachers' talk did differ in that they explained, modelled and 
monitored more often asking fewer questions, giving fewer instruc- 
tions and less feedback. In other words, teachers talked to the stu- 
dents more while asking the students for less talk. For all students, 
there was scant dialogue or instructional conversation. Teacher talk 
predominated in all the programs at two times the rate of student 
talk. When teachers and students did interact students' responses 
were frequently nonverbal or simple information recall statements. 

This evidence presented in the Ramirez study supports the per- 
sistence of the assessment or recitation model of instruction which 
minimizes social interaction and student language production. The 
'"script" of the recitation model consists of assigning text material to 
students, asking them to "recite" from it, most often through quiz, 
worksheet, or test, and assessing whether or not the student>s 
learned it. This teaching-by-assessment is in contrast to teaching- 
by-assistance, a model of teaching associated with the theoretical 
work of L.S. Vygotsky (Tharp & Gallimore, 1988). 

From Vygotsky and his disciples and elaborators, we are coming 
to know how teaching takes place in ordinary interactions of every- 
day life and results in the generation of higher order thinking. "The 
developmental level of a child is identified by what the child can do 
alone. What the child can do with the assistance of another defines 
what (Vygotsky) called the zone of proximal development.. ..It is in 
the proximal zone that teaching may be defined." In Vygotskian 
terms, teaching is good only when it 'awakens and rouses to life 
those functions which are in a stage of maturing, which lie in the 
zone of proximal development.' (Vygotsky, 1956, p. 278, quoted in 
Wertsch & Stone, 1985, Tharp & Gallimore, 1988, p. 4). In this re- 
definition of teaching, teachers assist students through their Zones of 
Proximal Development anticipating, selecting and maximizing the 
moments to assist student performance. 

Of the many ways to assist performance, dialogue, the ability "to 
form, express, and exchange ideas in speech and writing" is funda- 
mental to the development of thinking skills and "is the way parents 
teach their children language and letters.'* Dialogue or, in Tharp & 
Gallimore's term, the Instructional Conversation, occurs through 
"the questioning and sharing of ideas and knowledge that happens in 
conversation" (Tharp & Gallimore, 1988, p. 5). The Ramirez finding 
that teachers in the study generated few language production oppor- 




tunities for LEP students raises questions about teachers' knowledge 
base and skill level in regard to language development and its rela- 
tionship to cognition. In a Vygotskian conception of teaching and 
learning, the absence of dialogue or instructional conversation seri- 
ously constrains development of students' higher level thinking and 
complex learning. 

Fourth Recommendation 

Therefore our fourth recommendation for LEP teacher education 
program evaluation is that it identify and examine programmatic 
features and teachers' experiences important to language develop- 
ment. Language development features include teacher student ratio 
of talk, progression and level of student teacher talk, and student op- 
portunities for talk using school language and students' first lan- 
guage or dialect. Data collection strategies for language develop- 
ment include observation, informal records and student interactive 
journal entries, and other productions. Alternate data collection 
strategies for other programmatic features include descriptive sum- 
maries and samples of student materials. 

Examples ofPreservice and 

In-Service Program Evaluation 

The program evaluation experience of two programs, a Univer- 
sity of Hawaii alternative program, Preservice Education for Teach- 
ers of Minorities (PETOM), and the California New Teacher Project 
(CNTP), at the University of CaHfornia at Santa Cruz (SCCNTP), 
provides examples of veteran and preservice teacher education pro- 
gram evaluation. 

PETOM Preservice Program Experience 

In a state with a majority population of linguistic minorities such 
as Samoan, Tongan, Filipino, Laotian and Hawaiian Creole or pidgin 
dialect speakers, many students', and particularly those at-risk, sole 
experience with Staiidard American English occurs only in the school 
setting. The Native Hawaiian Educational Assessment Project de- 
scribed "Persons of Native Hawaiian ancestry" as those who "have 
suffered disproportionately from educational and social inequality for 
some time. Descendants of the original inhabitants of Hawaii find 
themselves at the bottom of indicators of success in modern America, 
and they are sometimes referred to as "strangers in their own land" 
(1983, p. 3). As a teacher educator and alternative teacher education 
program developed at the University of Hawaii, oral and written lan- 
guage development of young people was as central a curricular issue 
for teacher preparation as multicultural understanding and sensitiv- 
ity. 
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The faculty of an experimental teacher education program, titled 
Preservice Education for Teachers of Minorities (PETOM), was com- 
mitted to preparing teachers to provide at-risk minority students 
with experiences in language leading to the attainment of literacy 
and the ability to function in mainly verbal settings such as those of 
classrooms and, eventually, the workplace, community and society. 

This commitment was underscored by the inclusion of a semester 
course in Language Development within the program's two-year, 
field based curriculum. PETOM's efforts to evaluate its program 
quality and student development were collaborative producing and 
evaluation report each year for six years. Three features of PETOM, 
the program's origins, curriculum, organization and conceptual 
framework relate to the program's evaluation experience. 

First, PETOM grew from the work of the Kamehameha Early 
Education Program (KEEP) teacher consulting model. In KEEPs 
model of in-service teacher consultation, evaluation was continuous 
taking the form of weekly observation and feedback coUegially pro- 
vided by a peer consultant and quarterly data feedback provided by 
criterion-referenced testing. KEEP's in-service teacher training ef- 
fectiveness was evaluated by the use of standardized test scores of 
Hawaiian, part-Hawaiian students' reading skills. While undoubt- 
edly successful, KEEP'S in-service teacher training program invested 
3 to 5 years in teacher development. 

PETOM, as an outgrowth of KEEP, sought to capitalize on the 
preparation, practice, and reflection time available to preservice 
teachers during their professional training. A preservice teacher 
education program incorporating the principles of KEEP combined 
with extensive field experience in classrooms serving diverse stu- 
dents could facilitate preservice teachers' developmental course from 
novice to skilled. 

Secondly, PETOM's curriculum included generic methods em- 
phasizing integration of the content areas, language development, 
inquiry process, classroom management, child development and 
foundations. Faculty objectives for the course work included the 
translation of theory into practice and maximal modeling of instruc- 
tional strategies. Course work built directly on the novice teachers' 
field experiences. 

Thirdly, PETOM's faculty worked collaboratively to develop and 
implement the program. The Methods Instructors commented about 
their experience saying, 

"Traditional instruction for preservice teachers is scattered 
among different faculty members who rarely talk with one an- 
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other. No single group is responsible for seeing the preservice 
teachers through their entire program. As Eisner pointed out, 
university faculties may be even more isolated than elementary 
classroom teachers. Teacher educators rarely know what the 
other is teaching and rarely, if ever, observe one another teach. 
PETOM has provided an opportunity to break down this isola- 
tion" (Picard & Young, 1990, p.31). 

The program's theoretical framework, a Vygotskian conception, 
sensitized the faculty to the ways that teachers assist students to 
learn and teacher educators assist preservice teachers. The faculty 
struggled to understand the development of teachers' professional 
thinking and skills by the Vygotskian principles; as students learn in 
conversation with their teachers, teachers learn in conversation with 
their faculties and in the classrooms of their early field experiences. 
PETOM*s preservice teachers were assisted through their proximal 
zones, by their instructors and/or peers in program activity including 
classroom interactions, course discussion, interactive Journals and 
peer and/or instructor review of lesson videotapes. 

Again PETOM Instructors described it saying, 

"there are times when we may not seem to be clear as to whether 
we are talking about the education of teachers or the education of 
children. The truth is that many of the principles that follow ap- 
ply to both. Our expectation is that teachers will teach as they 
have learned" (Picard & Young, 1990, p. 31). 

As an experimental program within the College of Education, 
PETOM conducted annual program evaluations. Over the course of 
three cohorts of two years duration each, several evaluation designs 
were used. The findings were reported to the Dean, relevant College 
of Education committees, and other audiences. However, of the 
greatest importance was the feedback to the faculty for program de- 
velopment purposes. The faculty was responsive to positive and 
critical feedback provided directly by the students, revising course 
work requirements and on communication issues. All of the evalua- 
tion methodology used represented faculty interest in obtaining feed- 
back about developing their own and teacher performance compe- 
tence in conceptual and specific skill areas emphasized in the pro- 
gram. 

The measures used and listed in Appendix A included checklists 
of the College of Education for rating performance competence of the 
PETOM teachers. PETOM checklist data was compared to control 
group checklist data from the traditional program. Demographic 
data, such as ethnicity, age, grade point average and place of birth, 
were compiled demonstrating PETOM's commitment to diversity in 
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its recruitment and retention. PETOM students and graduates were 
interviewed about their program experience and asked to rate their 
program experiences and the program components. Cooperating 
Teachers were interviewed about their experiences in the program. 
Principals in the schools where PETOM students were hired as first 
year teachers were interviewed for their perspective on the PETOM 
teachers performance. Situational probes were devised to discrimi- 
nate between PETOM students and the control group for competence 
in program emphases such as language development and the use of 
culturally compatible teaching strategies. Open ended, stimulated 
recall interviews of the preservice teachers were a requirement of the 
field experience. Some of these were conducted and coded. Of these 
methods, the students' ratings, principals' comments, situational 
probes, and open-ended interviews will be presented and discussed. 

Student Ratings 

After the first cohort of PETOM students had taught, they v/ere 
given a survey questionnaire to rate the usefulness and relevance of 
their learning experiences in the program to their subsequent teach- 
ing. Their ratings demonstrated that the students considered their 
general undergraduate course work (in their case, the first two years 
of university general education requirements) barely relevant (or 2 
out of a possible 5 points) to their teaching preparation. In contrast, 
the field experience opportunity was considered most relevant with a 
rating of 4.5 out of a possible 5 points. The education course work 
was also very positively valued at just under 4.5. 

Immediately after the third cohort of students completed the pro- 
gi^am, they were asked to rate questions about their PETOM educa- 
tion. In the ratings, there is a positive trend in general. However, 
the pattern emei^ng from the rank ordering is one that indicates 
the graduates favored the more participatory kinds of experiences 
such as student teaching and observation-participation over the 
course work and the field experience seminars (Speidel, 1990, p. 77), 

While it is desirable to understand the perspectives of program 
graduates, the value of this type of feedback to the faculty and pro- 
gram is limited. These measures validate novice hunger for hands- 
on experience and give little information about teacher development. 
To obtain more useful data for their purposes, the faculty decided to 
obtain data from the principals hiring the program graduates. 

Principals^ Comments 

Overall the Principal's comments were positive describing the 
teachers as good, fine, excellent, and strong. The principals' negative 
comments included such statements as: (1) the teachers had difficul- 
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ties applying their knowledge due to lack of experience (3 times) and 
(2) the teachers need more relevant field experience (3 times). One 
principal commented on a teacher's unsmiling demeanor. In the nar- 
ratives, principals described teachers indicating their own criteria for 
good teaching. One example is: 

Principals^ Narrative Data Samples: 

"I think Bev is a really good example of an excellent teacher. She 
is just beginning now to take some responsibility for the rest of 
the faculty in sharing some of her ideas. She attended a work- 
shop where she had to come back and share what she had 
learned with the rest of the faculty and she did an excellent job. 
I look upon her as someone who will someday become Teacher of 
the Year. She is unassuming and sincere in her efforts. Bev is 
always open and eager to learn. I think either your program did 
a good job on her or else she is just a natural teacher. I think 
your progi^am should take some of the credit. Beverly is exem- 
plary." 

Joan and the other two PETOM graduates now teaching at my 
school are very enthusiastic and always willing to learn. They 
have a good knowledge base and are well equipped wdth effective 
teaching strategies although being new teachers they are still 
having difficulties applying them. 

They are middle-class teachers dealing with low-income, severely 
at-risk youngsters and this is an incredibly hard task. Therefore, 
they have had to make a lot of adjustment this first year and in 
many ways they are not yet fully equipped. I look at these teach- 
ers as slowly evolving and in two more years they will be top- 
notch. But they are never frustrated or depressed. They are 
lovely people who are trying hard and maintaining very good at- 
titudes despite the many obstacles that working with deprived 
youngsters bring. 

Working on the Leeward Coast is always a very frustrating expe- 
rience for very new teachers because most of them have not had 
the hands-on time with these kinds of kids. They have to first 
experiment with and weed through a multitude of teaching strat- 
egies before finding those that are most effective. These strate- 
gies must address all kinds of kids and not just those found at 
Kamehameha Schools or in town. Most young teachers have not 
had enough field experience in this area and it would really help 
if they spent time out here before graduating from school....! 
think your program is wonderful but your graduates need to 
spend more time in areas such as this before they can be truly 
competent new teachers. These teachers are growing this year 
and they are always willing to learn. I am sure they will evolve 
into first-rate teachers." 
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For the faculty, principals* comments were validating in that the 
PETOM students were generally successful as beginning profession- 
als in a school setting. This data was informative about contexts of 
school culture and community and revealed the principal's belief in 
the value of "hands-on experience" and the artistry of teaching. The 
data were less useful for increasing faculty understanding about the 
effectiveness of their course work. The teacher characteristics high- 
lighted in these comments, while important, were only generally re- 
lated to course work objectives. 

The faculty decided to design measures for collecting course work 
specific data from PETOM preservice teachers. Situational probes 
were constructed by the instructors for each course completed during 
the year of the evaluation. The written probes explored for the effect 
of the experience of PETOM on its preservice teachers by asking 
them to provide substantive responses when presented with written 
situations. It was hypothesized that PETOM student responses 
would be program specific and substantively richer than those of a 
control group of students from the traditional program. An example 
of a Situational Probes developed by the Language Development 
course Instructor follows: 

Situational Probe 

YouVe organized your second grade class into small instructional 
groups, but you ve noticed that the discussions youVe been hav- 
ing in these groups haven't been going as well as you'd like. The 
students typically give one word overly brief responses. You'd 
like them to give lengthier answer and elaborate on each other's 
responses. 

a. What will you do in an effort to improve the discussions? 

b. What are your reasons for suggesting these actions? 

Sample answers given by PETOM students: 

1. Ask open-ended questions, questions that allow for more diver- 
gent responses. Ask student to elaborate answers. Open/diver- 
gent questions require more than one word responses and also 
give teacher and students a "forum" for extending answers - ask 
students to explain response, add to it. 

2. I would try to state questions in such a way that requires more 
than one word answers. Questions could begin with "Wliy do you 
think...?," "How...? " "What could/would happen if..?," "What do 
you think about...? ...However, I think that all this must be 
taught to the students; they need to be guided through a discus- 
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sion. At first, the teacher must take the lead but the ultimate 
goal would be for the teacher to only help focus and clarify in a 
discussion. 

3. I would draw on the children's background and experiences. By 
drawing on experiences, the discussion becomes more interesting 
and students will feel comfortable in contributing to the discus- 
sion. 

Sample Non-PETOM Responses: 

1. Ask the students to answer in a sentence form. 

2. Make rules regarding the discussions: Everyone needs to re- 
spond with at least three sentences and respond to someone 
else's contribution to the discussion. By providing rules to the 
discussion and requiring each child to participate this would help 
to increase the length of the answers and elaboration. 

3. To improve the discussions I would try having one large group in 
which students raise their hands to respond to questions or an- 
swers. The reason for suggesting this is that this particular 
group may not work well in groups. As a group (large one), the 
students may respond better instead of in smaller groupings. 

The data collected with the situational probes distinguished the 
PETOM students from non-PETOM students. This was not the case 
with every probe but, in general, the data indicated that the students 
were appropriating conceptual material featured in the program's 
course work. However, the faculty noted that this paper and pencil 
task did not reveal the students' reactions during the acts of teach- 
ing. The latter appeared to be the most critical information about 
the program effects and the most promising for understanding the 
teacher development process. In the interviews to be discussed next, 
the faculty found those data, and they were revealing indeed. 

Interview Data 

Part of PETOM's field experience required that the preservice 
teachers videotape themselves teaching. In interviews about their 
videotaped lessons, the preservice teachers began to disclose their 
progress through a rigorous developmental process of appropriating 
and applying course work concepts. 

A preservice teacher describes her experience: 

**Sometimes, too, when you're told things and when you actually 
do it, it comes out different. That's what I'm finding out, too. 
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They cannot always be done like how (youVe told in courses) or 
there isn't any one best way to do something. Sometimes, too, it 
just depends on the situation and what's going on. It's really 
hard." 

Although she arranged her teaching activities according to 
course work guidelines and recommendations, the preservice teacher 
found that the "situation" or the social conditions of teaching com- 
pounded the challenge. Such occasions of purposeful social activity, 
in Vygotsky's conception, are the basis for forming and transforming 
thinking. Such data from the perspective of the participant reflects 
the process of individual development and has the potential to inform 
program development and expand the knowledge base. 

Another preservice teacher said: 

"I don't think I really thought too much about the kind of prepa- 
ration and the time it would take to do research and to absorb it 
and understand it enough to teach it to someone, to break it 
down into steps. That, I kind of had a hard time adjusting to the 
students' need, and how you can better move them to the point, 
more guided." 

This statement describes the challenge that "guided" or teaching 
that assisted performance makes on the teacher. The preservice 
teacher in using terms like "adjusting to the students' need" and 
"how you can better move them to the point" is operationalizing her 
understanding of the zone of proximal development. She considers 
the amount of assistance the student requires based on what she be- 
lieves he can do with assistance that he cannot do independently. In 
her words, she will "break it down into steps" in order to assist the 
student to the "point" or lesson goal. 

Typically, the complexity of sophisticated interaction between 
teacher and students is camouflaged from the novice. It is in the ac- 
tivity itself that the challenge of assisting students to learning goals 
becomes crystal clear to the preservice teacher. Tharp and Gallimore 
emphasize that "Assisting performance through conversation re- 
quires a quite deliberate and self-controlled agenda in the mind of 
the teacher, who has specific curricular, cognitive, and conceptual 
goals" (Tharp & Gallimore, 1988, p.5). Measures that collect data 
about participants' experience from their point of view during the ac- 
tivity of teaching produce data of use to faculties for understanding 
course effect and the process of teacher development. 

Teacher education and professional development programs em- 
phasizing performance assistance for linguistically and culturally di- 
verse students, heighten preservice teachers' sensitivity and respect 
for their students' linguistic and cultural differences. LEP programs' 
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aim to assist the performance of preservice and veteran teachers by 
equipping teachers with appropriate strategies and techniques for 
effectively educating diverse students. These strategies are designed 
to remove many of the barriers to communication by adjusting the 
dynamics of teacher student interaction to be compatible with stu- 
dents' preferences. 

Teachers' application of such strategies can be assisted through 
observation and feedback as well as interview. In the following in- 
terview excerpt, a preservice teacher describes her use of one strat- 
egy for discussion labelled, "talkstory" and she also describes her 
feehngs. She says: 

"The talk stoiy format... seems to be more comfortable, share 
whatever theyVe thinking at the moment...! like it. I don't mind 
it in a small group. I think I can handle it better but I know 
when it gets bigger, 5 is very good for me and if it gets to like 10, 
it gets harder to manage and I have a harder time hearing. I 
have to cue a lot more about I can't hear so and so talk. Some- 
body else is talking at the same time I have to keep cueing and 
reminding them about it. 1 can only hear so many people at a 
time. But I feel pretty comfortable with it. I enjoy it and I know 
the kids enjoy it. So it's a lot more comfortable and natural.... 
cause they seem to participate more. They get more excited 
about it and motivated." 

The information from the interview data included information 
about teacher affect, teacher attitude toward the students, and the 
teacher's progress appropriating the strategy. Her discussion also 
revealed her use of another technique, cueing, which she was apply- 
ing simultaneously with talk-story. Her facility with the "talk story" 
technique and her comfort level in using it indicates her progress 
from novice to skilled professional. This is valuable feedback for fac- 
ulty and program developers. 

The interviews provide rich data suggesting linkages and inter- 
actions among the program, the knowledge base, and the teacher de- 
velopment process. Analysis and interpretation of this data within 
the context of the total program evaluation holds promise for inform- 
ing program development. In-depth interview data promises to re- 
veal not only more about processes of teacher development but more 
about progi-am development processes. Discovery of program's 
sources of influence, mechanisms of information exchange, and de- 
gree of openness to research reports and the established knowledge 
base is as much faculties' responsibility as the understanding of the 
specific effects of course work on teacher performance. 
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PETOM Experience Summary 



The progress of this faculty through program evaluation was col- 
laborative, shared aixd based on their common understandings about 
the importance of obtaining data feedback that was useful for pro- 
gram development within the program's conceptual framework. 
They designed a variety of measures for examining the effect of their 
own teaching and they struggled to understand the meaning of the 
data generated by their measures. In collaboration, the faculty de- 
signed measures for various program constituencies. In continuing 
collaborative evaluation work, these measures, constituencies and 
other contexts will expand in response to faculty's need to obtain use- 
ful data for program development. Importantly, they were respon- 
sive to evaluative feedback by redesigning their own course work and 
refining program experiences. 

University of California at Santa Cruz (UCSC) In- 
Service Program Experience 

In the past 20 years, teachers have seen an explosion of new 
ideas and programs for improving classroom instruction. Extensive 
in-service training initiatives have become the traditional vehicle for 
conveying new pedagogical strategies. Unfortunately, teachers have 
typically been viewed as recipients rather than as decision-makers or 
active participants in staff development programs. Staff develop- 
ment is often seen as "training" or "in-servicing^ in which experts 
teach teachers predetermined instructional methods. This raises an 
evaluation question, that is, how can veteran teachers be helped to 
implement new strategies for working with LEP students? 

The body of research on in-service training indicates certain 
characteristics that make it more effective and calls for new ways of 
looking at retraining teachers. The work of Glickman suggests that 
teachers need not be trained but rather be given the tools for deter- 
mining their own instructional priorities. Components that allow 
teachers to work together and make decisions about planning, imple- 
menting, and evaluating instruction (Glickman, 1990) are highly de- 
sirable. He found teachers became more thoughtful and resourceful 
about teaching. 

Teacher education and in-servicing strategies such as peer coach- 
ing, simulations, demonstrations, performance feedback, interactive 
journals, and mentoring are currently receiving attention in the lit- 
erature. These in-service strategies and recent programs like 
teacher induction enhance the opportunity for professional develop- 
ment within the social, interactive context of teaching activity. AI- 
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though teacher induction programs vary in their design, the overall 
goals of these programs have been to provide ongoing support and 
assistance to beginning teachers as they enter the profession, to im- 
prove teacher effectiveness, to increase retention in the profession 
and to promote the professional and personal well-being of new 
teachers, 

Veenman (1984) suggested that novice teachers need both peda- 
gogical assistance and psychological support. This is similar to rec- 
ommendations from developmental theorists such as Furth (1981) 
and Vygotsky (1978), They point out that a supportive atmosphere is 
necessary if learners need to master new and complex thought and 
action. 

As Gherke points out, traditional quantitative methods are inap- 
propriate for trying to understand these relationships. Recent evalu- 
ation reports about induction programs suggest that useful data re- 
garding teacher networking, nurturing relationships and complexi- 
ties of teacher development are available through qualitative meth- 
ods (Gherke, 1988), 

UCSC/Santa Cruz County New Teacher Project 

Based on research recommendations on effective in-service train- 
ing, the University of California/Santa Cruz County New Teacher 
Project designed an interactive, collaborative program for new 
teacher support, staff development, and professional growth for vet- 
eran teachers. A program titled, the Santa Cruz County New 
Teacher Project was developed in 1988 by the University of Califor- 
nia at Santa Cruz Teacher Education Program in collaboration with 
the Santa Cruz County Office of Education and seven school districts 
in the county. In this consortium, communication and collaboration 
occurred across districts and institutional boundaries. The consor- 
tium members determined program philosophy, components, and on- 
going evaluation. 

During its first three years the project has served 155 first year 
K-8 teachers with 105 teaching in bilingual classes; 78 of these teach- 
ers are graduates of the UCSC Teacher Preparation Program. The 
Santa Cruz County New Teacher Project supports beginning teach- 
ers' efforts to translate what they have learned in their preservice 
preparation into classroom practice. After nearly three years, the 
Santa Cruz County New Teacher Project has lost only five of the 155 
teachers served. 

In this project, the evaluation component was designed by pro- 
gram faculty,and included both quantitative and qualitative meth- 
ods. Measures, listed in Appendix B, such as standardized inter- 
views, questionnaires, journal entries, videotape feedback, advisor 
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logs, self-assessment forms and weekly observations were systemati- 
cally used by all project participants. The faculty reviewed data and 
responded to feedback at weekly meetings of the bilingual advisors 
and the director. For three years, the program development was 
guided by evaluation data. What follows is a description of the 
project and an overview of the findings. 

The project serves seven school districts in Santa Cruz County 
which is a rapidly growing area with an ethnically and linguistically 
diverse student population of about 33,000 K- 12 students. The 
Pajaro Valley Unified School District, the largest in the county, is 
one of the most hnguistically impacted in CaUfornia with 40 percent 
of its students being limited-English proficient. Ninety-four percent 
of these LEP students are Spanish speaking, and of this group, 77 
percent are migrant. 

The Santa Cruz County New Teacher Project (SCCNTP), one of 
37 pilot projects of the California New Teacher Project, is funded by 
the State Department of Education and the Commission on Teacher 
Credentialing. Five exemplary biUngual teachers - Novice Teacher 
Advisors - are the cornerstone of the Santa Cruz County New 
Teacher Project. The advisors, who specialize in all aspects of bilin- 
gual-multicultural education, are hired to work with new teachers 
for the entire year under the guidance of a UCSC Project Director. 

The advisors, whose case load was fourteen bilingual teachers, 
worked with each new teacher two hours per week both in and out of 
the classroom setting. The advisors' time in the class was spent do- 
ing demonstration lessons in both English and Spanish particularly 
in language and Hteracy, observing and coaching, assessing students, 
videotaping lessons, providing release time, responding to interactive 
journals, and assisting with problems as they arose. Time outside 
the classroom was spent on planning, gathering bilingual and cultur- 
ally relevant resources, problem solving and reflection, and general 
support and encouragement. By being familiar with the students in 
the class, the overall curriculum plan, and the class structure and 
organization, the advisor was able to provide new teachers with con- 
text specific assistance. 

The overall philosophy of the project is that teaching is complex 
(Good and Brophy; Shavelson, 1983) and that the process of becom- 
ing a teacher involves career-long or life-long learning. The project 
recognizes that new teachers enter the profession at different devel- 
opmental stages and with individual needs. In a non-evaluative and 
supportive manner, the advisors help each new teacher develop an 
individualized plan to address their specific goals and needs. From 
week to week the advisor and the new teacher work together to 
strengthen the new teacher's program with the advisor responding to 
the new teacher's zone of proximal development. Each new teacher 
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is a member of a team which includes the advisor, the peer resource, 
and the site principal. 

In addition to the ongoing coaching, new teachers received five 
days of staff development training aimed at meeting the identified 
needs of the new teacher and their LEP students. The first in-ser- 
vice on classroom management and organization emphasized coop- 
erative learning strategies and heterogeneous groupings. The sec- 
ond inservice on Language Development addressed first- and second- 
language acquisition, strategies for creating natural language oppor- 
tunities, methods for integrating language into all areas of the cur- 
riculum, and thematic planning. The ReadingAVriting Connection 
focused on ways to create a biliterate learning environment, rich in 
print and conversation. 

The daha collected following each session was positive. It indi- 
cated that the new teachers felt that they had received information 
which was immediately applicable and manageable within their first- 
year context. The weekly coaching that followed supported new 
teachers in their efforts to acquire new strategies. Two days of re- 
lease time were spent with the advisor observing exemplary bilingual 
teachers, discussing the observations, and planning curriculum. A 
monthly seminar series provided an opportunity for networking and 
reflection. 

Evaluation 

As a result of their intensive involvement, the advisors developed 
a unique and powerful coUegial relationship with each of their new 
teachers. Both midyear and end of the year evaluations from the 
new teachers used such descriptors as "saint," "guardian angel," 
"friend, and "co-teacher" to describe this relationship. When given 
an opportunity to describe the most beneficial aspect of their work 
with the advisor, new teachers,(male and female/ages ranging from 
22-50) consistently emphasized emotional support. When new teach- 
ers in the Santa Cruz project were asked "Who has been helpful in 
dealing with the challenges you've faced?" the New Teacher Project 
was the most frequently named source of support (Drury, 1991). The 
level of enthusiasm for the project increased throughout the year 
with teachers repeatedly attesting to the value of the help they re- 
ceived from their advisors and other project personnel. 

New teachers and their advisors kept an ongoing interactive 
journal. The journal entries present the types of questions new 
teachers pose, the frustrations they face, and the depth of the rela- 
tionship between the advisor and the advisee. The following excerpt 
from a third grade bilingual teacher shows her taking charge of her 
own professional development. "For our next meeting Td like to 
meet after school, if possible, to plan a math schedule like you 
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showed us at the in-service. Fd like to bring in some other strands 
from the math framework Uke your model does. It is obvious to me 
that many of the students need additional opportunities for hands on 
activities." 

A second grade bilingual teacher wrote: "I haven't been happy 
with my writing program. I had great plans at the beginning of the 
year. I guess where it bogged down was with my frustration with 
trying to attend to each student's writing needs. It wasn't possible 
for me to get around individually and actually do much teaching. 
Because I was disturbed by all these factors I tended to avoid writ- 
ing, especially revision and editorial stages. Maybe you could dem- 
onstrate a writing lesson for me sometime." 

Afler the demonstration lesson he wrote: "I loved the writing les- 
son. This is exactly what I've been missing, some motivating mate- 
rial to be creative with. They really do need some motivating struc- 
ture to bounce off from. I guess its asking too much for them to cre- 
ate from a vacuum, which I often have done. As beginning writers, 
they need to play with vocabulary in this way, one step at a time. I 
also read your comments on their papers. Makes me see how infre- 
quently I give positive feedback. It's so easy to write "fantastic idea." 
Why don't I do it more?" 

The project director and the advisor had regular contact with the 
site principals. The principals' perspective about the SCCNTP was 
collected through both interviews and questionnaires. All of the 30 
principals interviewed clearly felt that the most important part of the 
project was the weekly classroom visits by the advisor. One of the 
participating elementary principals spoke to the program's effects: 

"The project is supporting new teachers in all the ways that prin- 
cipals would like to but never have the time to do. When I com- 
pare the first year teachers of previous years to this year's new 
teachers, it's apparent to me that this group is much further 
along in developing their programs. In less than a year, they are 
doing what it took other teachers three years to accomplish. I 
attribute this to the close working relationship with the veteran 
teacher." 

These quotes from principals strongly suggest that nurturant re- 
lationships within classroom contexts support teacher growth. The 
advisors' years of experience in bilingual classrooms and their train- 
ing enabled them to diagnose problems, provide options, and allow 
the new teachers to remain in control of decision making while being 
extremely sensitive to the developmental needs of each new teacher. 

Again, a principal at a middle school stated: "We saw more 
growth in our first year teachers than we have ever seen before. By 
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working as a team with a veteran teacher, they are giving the kids in 
those classrooms a far better education than they would get if that 
link wasn't there." 

One of the unanticipated benefits to veteran teachers at the 
school site was the spillover effect of the collegiality and collaboration 
being modeled by the new teacher and the new teacher advisor. 
When 50 new teachers were asked, "What effect, if any has the New 
Teacher Project had on your staff?" all but five noted a positive 
change. The responses included increased networking and sharing 
of resources, a willingness to share ideas and strategies, a more pro- 
fessional staff, an openness to new ideas; individual staff members 
have been rejuvenated and motivated to rethink strategies, innova- 
tive methods are spreading throughout the school, greater respect for 
new teachers, and more sensitive to the demands and pressures of 
first year teachers. Ten of the teachers mentioned that veteran 
teachers at their school sites often approached them or their advisors 
to be a part of their sharing or to receive copies of resources the advi- 
sors brought. This gave new teachers a boost in self-esteem as they 
could now be givers rather than always "takers," One new teacher 
stated in May, "It seemed like the veteran teachers used to run away 
whenever they saw me coming. But as I've acquired new materials 
and teaching strategies they often ask about my new ideas. I don't 
feel like a parasite any longer." 

The advisors in the SCCNTP received extensive staff develop- 
ment training in cognitive coaching, clinical supervision, communica- 
tion skills, the needs and developmental stages of new teachers, and 
effective strategies for working with LEP students. As part of the 
evaluation component, the advisors were asked to describe what 
they'd learned in their two years of working with the project. What 
follows are examples highlighting the reoccurring themes that 
emerged. 

I feel a particular benefit from all I have learned about question- 
ing as a means to help teachers think about their teaching and 
consciously control it through their own decision making. I think 
that the non-evaluative nature of my relationship with new 
teachers has allowed me to freely explore the possibilities of nov- 
ice-veteran teacher interactions. We didn't have to prove any- 
thing together. This experience has caused me to develop a bet- 
ter understanding of the dynamics of change as it applies to 
teachers. 

Participating in this Project has me fully convinced that the 
flourishing of the profession will depend in a large part on the 
opening of our doors to our peers and that teachers must control 
the process. It can't be done to them. 




By seeing so many different ways to make teaching work I have 
become more accepting of differences in style and approach. We 
teach a diverse student population yet I had fallen into the pattern of 
seeing a few instructional approaches and teaching styles as "best." I 
have learned that you have to begin with a relationship and then you 
have the opportunity to dig deeper through questioning to arrive at a 
closer understanding of the educational decision making that went 
into creating a learning activity. It is then that you begin a profes- 
sional dialogue, not with the purpose of persuasion but with an invi- 
tation for a thought provoking exchange - and with this kind of em- 
powerment comes change. 

When I faced a problem in helping a new teacher, I could always 
problem solve with the team of advisors. Together we came up with 
questioning strategies and new approaches that enabled us and the 
new teacher to understand better. We were able to "shadow" one an- 
other in pairs, peer coaching each other at work with a new teacher. 
As a member of this coUegial network of peers, I was able to grow in 
my own teaching. 

As a member of a team of experienced teachers, working together 
to support new teachers, I had weekly opportunities to brainstorm, 
problem solve, share professional expertise and resources with my 
peers. We felt respected for our talents, skills, strengths, and even 
for our areas of weakness. Our director always consulted members 
of the team in decision making and she was able to keep the dialogue 
open and constructive on a regular basis. This freedom and open- 
ness empowered us to take responsibility for our own professional- 
ism. I attribute the success we had in supporting new teachers to 
this feeling of safety in the community we created among ourselves. 

This project is also externally evaluated. The Southwest Re- 
gional Laboratory is completing a three year evaluation study of the 
California New Teacher Project. Preliminary findings conclude that 
in addition to dramatically affecting retention, high intensity support 
efforts as provided by the Santa Cruz County New Teacher Project 
also greatly enhance teacher performance, especially in the area of 
multicultural education and working effectively with diverse popula- 
tions. SWRL has also found a positive relationship between the level 
of support offered to new teachers and the amount of time new teach- 
ers actually engage their students in academic learning tasks. In 
fact, those new teachers who had high intensity support came close 
to providing as much academically engaged time for their students as 
do experienced teachers. New teachers in the high intensity models, 
where they received support and training, reported that the CNTP 
was important to their success. 

Although these quantitative data gathered by SWRL have been 
valuable, another kind of data was also needed. We needed to study 



the relationship between the advisor and the advisee in an effort to 
understand the kinds of interactions that support growth and devel- 
opment in new teachers. Upon reviewing the interactive journals, 
the advisors were eager to try alternative approaches and/or modify 
their support based on new teacher needs. The qualitative data in 
these journals and interviews highlighted the different developmen- 
tal stages that new teachers go through as well as the type of assis- 
tance they sought thi'oughout the year. The advisors had some work 
to do on their own acceptance and tolerance. 

Program responsiveness also extended to the role of the advisors. 
Most advisors began with the attitude that teachers teach in similar 
ways with similar methods. In reality, diversity permeated the en- 
tire program including evaluation - different teaching styles, with 
teachers at different developmental stages, advisors with varying 
levels of knowledge regarding coaching and biliteracy strategies, 
principals with different styles and seven districts with different pri- 
orities. This openness or embracing of diversity was the heart of the 
evaluation process. This diversity is illustrated by the questions that 
continually emerged. How can we help new teachers with limited 
Spanish who have been assigned to a bilingual class? How can we 
set up literature groups in a class where management is a major 
problem? What policy implications does our research design and 
data have for teacher education and in-service programs? These 
questions were best answered by coUegial teams that gave teachers 
opportunities to take control of their own professional development 
and to create non-judgmental strategies for sharing strengths and 
weakness. 

New teachers in the project opened their class up to weekly ob- 
servation without fear or intimidation. By building the new teachers 
into the evaluation design, their talents and professional status were 
acknowledged. This participatory model of evaluation means not 
"doing evaluation to" progi'am participants but including them in all 
aspects of the design, implementation, analysis, evaluation and pro- 
grammatic modification. 

Fifth Recommendation 

Our fifth recommendation follows from the participatory evalua- 
tion used in both the Hawaii and California progi'ams: Engage pro- 
gram participants in program evaluation design. Collaboration on 
evaluation design produces useful, substantive data that validate 
program operations and stimulates program responsiveness. A par- 
ticipatory model of evaluation means bringing program participants 
into the evaluation process from its inception. 
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UCSC/Santa Cruz County 

New Teacher Project Summary 

The collegia! team that was established among all the partici- 
pants, advisors, new teachers, principals, peer resource, faculty and 
the director, allowed for and encouraged an open forum for dialogue, 
continuous reflection, and evaluation. The program focused on advi- 
sors being sensitive to the Vygotskian principles in working with 
new teachers. The multi-modal evaluation with interviews, ques- 
tionnaires, interactive journals, advisor logs, videotaping, and self- 
assessment forms were directed toward the constant rethinking and 
enhancement of the program. Participants viewed the evaluation 
process as a positive opportunity for professional growth. When the 
project began in 1988 little was known about the needs of new teach- 
ers. It has been through the participation of this collegia! team in 
the evaluation process that we have gained in-depth awareness and 
insights into the experiences of new teachers serving LEP student 
populations. New questions arise and the process of evaluation con- 
tinues. 



Summary of Recommendations for Evaluating 
LEP Preservice and In-Service Programs 

In this paper, the teacher education program evaluation litera- 
ture, the findings of recent research on effective teaching and learn- 
ing models for linguistic minorities, and the experiences of two pro- 
grams, one preservice and one in-service, have been discussed. In 
both programs, a wide range of data were collected in an exploratory 
manner capturing the perspectives of many of the participants. The 
programs' evaluation designs sought data about process level issues 
as well as competency based teacher performance exploring the rela- 
tionship between what programs actually do with the knowledge 
base and how teachers develop. Both programs depended on this 
evaluation data to refine operations and proceeded in a collaborative, 
activity based manner. 

The experiences of these evaluated programs though clearly still 
evolving reinforce many recommendations in the literature. Build- 
ing on the literature, the experiences of both programs, and effective 
teaching and learning strategies for LEP students, we conclude with 
five recommendations presented as follows and in Appendix A: 
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Recommendations for Evaluation ofLEP Preservice and 
In-Service Teacher Education Programs 

1. Identify and document LEP teacher education and in-service pro- 
grams using program evaluation. 

2. Identify and document program responsiveness to evaluation 
feedback. Evaluation methodology designed to yield relevant, 
substantive, and useful information for program developers and 
faculty is most likely to result in program responsiveness to feed- 
back. 

3. Use broad based methods and measures to obtain multiple per- 
spectives on LEP teacher education program experience and ef- 
fect particularly those of the specific communities, cultures, and 
constituencies served. 

4. Design LEP teacher education program evaluation to examine 
programmatic features important to language development and 
program responsiveness to new knowledge in the field. 

5. Engage program participants in program evaluation design. Col- 
laboration on evaluation design produces useful, substantive data 
that validates program operations and stimulates program re- 
sponsiveness. A participatory model of evaluation means bring- 
ing program participants into the evaluation process from its in- 
ception. 

Given the dearth of evaluation activity in LEP teacher training 
and in-service and the crisis the nation faces in addressing the needs 
of hnguistic minorities, program evaluation demands prioritization. 
At this time, the linkage between the expanded knowledge about ef- 
fective education for LEP students and teacher education program 
development is unclear. Additionally, the relationship between what 
programs actually do and how teachers develop in them is unre- 
ported. The effect of program operations, teacher development pro- 
cesses, and program's linkage to knowledge base are integral sources 
of information and influence for programs committed to improving 
teaching practice for LEP students. We need to know much more 
about these processes and program evaluation as one means to this 
goal. 

Evaluation can be a systematic and regular practice that is an 
essential component of the program development process, well worth 
the resources required. By engaging in self-examination, programs 
themselves enter a developmental process, learning as they teach. 




440 



Appendix A 



PETOM Evaluation Measures 

• Checklists for rating performance competence of the PETOM 
preservice teachers and controls 

• Demographic profiles of ethnicity, age, grade point average, and 
place of birth 

• Preservice teacher interviews about program experience 

• Cooperating Teacher interviews 

• Rating scales 

• Principals' Interviews 

• Situational Probes to discriminate between PETOM students and 
the control group for competence in program emphases such as 
language development and the use of culturally compatible 
teaching strategies 

• Open Ended Stimulated Recall Interviews about preservice 
teachers' experiences in the activity of teaching 



PETOM PrincipaUs Interview Excerpt 

The PETOM graduates "...are very enthusiastic and always will- 
ing to leam. They have a good knowledge base and are well equipped 
with effective teaching strategies although, being new teachers, they 
are still having difficulties applying them. 

They are middle-class teachers dealing with low-income, se- 
verely at-risk youngsters, and this is an incredibly hard task. There- 
fore, they have had to make a lot of adjustments this first year and, 
in many ways, they are not yet fully equipped. I look at these teach- 
ers as slowly evolving, and in two more years they will be top-notch. 
But, they are never frustrated or depressed. They are lovely people 
who are trying hard and maintaining very good attitudes despite the 
many obstacles that working with deprived youngsters brings. 

Working on the Leeward Coast is always a very frustrating ex- 
perience for very new teachers because most of them have not had 
the hands-on time with these kinds of kids. They have to first experi- 
ment with, and weed through a multitude of teaching strategies be- 
fore finding those that are most effective. These strategies must ad- 
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dress all kinds of kids and not just those found at Kamehameha 
Schools or in town. Most young teachers have not had enough field 
experience in this area, and it would really help if they spent time 
out here before graduating from school. ...I think your program is 
wonderful, but your graduates need to spend more time in areas 
such as this before they can be truly competent new teachers. These 
teachers are growing this year and they are always willing to learn. I 
am sure they will evolve into first-rate teachers." 



Appendix B 

University of California at Santa Cruz Measures 

• Standardized Interviews 

• Questionnaires 

• Journals 

• Videotape Feedback, 

• Advisor Logs 

• Self-assessment 

• Weekly observations were systematically used by all project 
participants. 

University of California at Santa Cruz 
New Teacher Project 

Examples of 

Program Responsiveness to Evaluation 

Based on the data from the first year, the program modified its 
support to new teachers. The First Year Orientation, focusing 
heavily on planning, classroom management and bilingual curricu- 
lum development was revised to be sensitive to the developmental, 
needs of new teachers. 

In response to data indicating new teachers' increased need for 
support, the project recruited exemplary, bilingual teachers who 
could combine their expertise in teaching with nonjudgemental, sup- 
portive interpersonal skills. 
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Based on feedback from new teachers, the seminar series dra- 
matically changed over the three years. Seminars moved from a cur- 
riculum emphasis to an open forum, enhancing networking, collabo- 
ration, and problem solving. 

Advisors began by expecting teachers to teach in similar ways 
with similar methods. They became responsive to different teaching 
styles, teachers at different developmental stages, principals with 
different styles, and seven districts with different priorities. 
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Response to Dalton and Moires Presentation 



Lynn Malarz 
Association for Supervision and 
Curriculum Development, Virginia 

As an educator very interested in professional development and 
bilingual education, I would like to compliment the authors on their 
papers and the research that went into writing them. It is an honor 
to be a discussant and be part of this symposium. 

First, let me make a few brief comments about the paper, then I 
would like to expand on certain aspects of the paper that research 
has shown to be important components of teacher training. I whole- 
heartedly agree with the authors that teacher training programs in 
general are not widely evaluated and it is very unfortunate that the 
evaluation of LEP teacher training and in-service programs is so ne- 
glected at a time when so many states are begging for bilingual 
teachers. For example, in its August newsletter, the California Ad- 
ministrators Association stated thai at the beginning of this year the 
state would be short 14,000 bilingual teachers. 

I can further say that I do not disagree with any of the points 
that the authors have made in their papers; the papers presented 
many pertinent issues regarding teacher education. Thus, I invite 
you to walk down another path with me, one that will expand on 
parts of the training I see as very crucial components in teacher 
training. If it is true, as David Berliner (1984) states, that teaching 
is a constant stream of decisions, and any teacher behavior used is 
the result of a decision, either conscious or unconscious, then as edu- 
cators working with teachers, we need to understand how to help 
teachers make the decision that will promote maximum student 
learning. By looking at the work of many researchers in the field of 
education, such as David Berliner Teacher "Executive Processes," 
Madeline Hunter "Teaching as Decision Making," Robert 
Goldhammer "Clinical Supervision," Art Costa and Bob Garmston 
"Cognitive coaching," and many more, (the authors referred to advis- 
ers being in-serviced in the cognitive coaching model as well as oth- 
ers), we can begin to understand how an individual can become an 
autonomous teacher - teachers who: 

• Act With Intentionality 

• Generate and Choose from Alternatives 

• Use Precise Language 

• Take Responsibility 

• Monitor, Reflect Upon, and Learn From Experience 
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• Align Behaviors with Values 

• Activate Community 

Let's look at Profiling the Teacher (see Appendix A) and the 
myriad of components that go into the profile. By looking at this, it 
can be said that "Effective supervision is defined as a set of strategies 
designed to enhance the teacher's perceptions, decision, and intellec- 
tual functions," (Costa, p.l7, 1991). 

Following are some ways to work with beginning teachers as well 
as veteran teachers to help enhance their intellectual skills, which 
contribute to educationally sound decision making. 

1. Cognitive Coaching- , which I have eluded to, is one way. It is a 
way that supervisors, advisors, coaches and others can work with 
teachers to help them with the coaching functions in the four 
phases of the instructional process (see Appendix A). 

2. Peer Coaching is another method that has been shown to in- 
crease collegiality and improve teaching. Peer coaching is a confi- 
dential process through which teachers share their expertise and 
provide one another with feedback, support, and assistance for 
the purpose of refining present skills, learning new skills, and/or 
solving classroom-related problems. 

Any type of coaching system has preconditions that should be 
considered before implementing the program, such as: 

There must be a general perception on the part of the people in- 
volved that they are good but can always get better; they can al- 
ways improve what they are doing. This general orientation has 
been found to characterize effective schools. 

The teachers and administrators involved must have reasonable 
level of trust; they must be confident that no one is going to dis- 
tort the situation in any way. 

There must be an interpersonal climate in the school that con- 
veys the sense that people care about each other and are willing 
to help one another. 

Research has also shown that to have meaningful staff develop- 
ment - programs that become institutionalized - schools need to 
have teachers participate in ways that ensure transfer of learning 
(see Appendix A). Further benefits of coaching programs have also 
been documented: 

• Better understanding of teaching 

• Improved self-analysis skills 

448 



• Improved sense of professional skill 

• Renewal and recognition 

• Increased sense of efficacy 

• Increased collaboration/collegiality 

• Improved teaching performance 

• Increased student growth/development 

Now one can ask, "what is the purpose to all this?" I know it has 
been verbalized these last few days at this symposium, but let me 
again state what I feel is necessary in bilingual education teachers 
need to increase their responses to students that will support and ex- 
tend student thinking and learning. By helping teachers learn dif- 
ferent strategies (cognitive coaching, peer coaching, etcetera.), 
higher order thinking skills can be attained. 

Research has found the manner in which teachers respond to 
students has great influence on the student. Different researchers 
have documented (Lowrey, 1990) that the way teachers respond has 
a greater influence on students' thinking than what the teacher asks 
or tells them to do. Students are constantly anticipating how their 
teachers will respond to their actions. Thus, the way teachers re- 
spond to students seems to exert greater influence than the teachers' 
questioning. It has also been found that teachers' responses have a 
great deal of influence on the development of students' self-concept, 
their attitude toward learning, their achievement, and their class- 
room rapport. 

Let me quickly review response behaviors of teachers - teacher 
initiated questions and directions that elicit thinking and learning 
(See Appendix A). 

Last, I would agree with the authors that teacher training and 
in-service needs systematic and regular evaluation. My hope would 
be that more programs not only have approaches and processes simi- 
lar to UCSC/Santa Cruz County New Teacher Project and the Uni- 
versity of Hawaii's Preservice Education for Teachers of Minorities 
but also incorporate the research that is constantly emerging on 
higher order thinking skills. Skills that can enhance a student's 
learning; skills that will help the student become a life-long learner. 
The Greeks had a word for it: Paideia: 

A society in which learning, fulfillment, and becoming human 
are the primary goals and all its institutions are directed toward 
that end. The Athenians designed their society to bring all its 
members to the fullest development of their highest powers. 
They were educated by their culture — by Paideia. Self-develop- 
ment and the promotion of lifelong learning is the "central 
project" of society, 

3o 
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Appendix A 



THE MANNER IN WHICH TEACHERS 
RESPOND TO STUDENTS HAS GREAT 
INFLUENCE ON THE STUDENT, 



TEACHERS NEED TO ENHANCE COGNITIVE 
LEVELS OF CLASSROOM INTERACTION, 



Gathering and Recalling Information (Input) 

To cause the student to input data, questions and statements are 
designed to draw from the student the concepts, information, feelings 
or experiences acquired in the past and stored in long or short term 
memory. 

Some verbs that may serve as the predicate of a behav- 
ioral objective statement are: 

completing matching 

counting naming 

defining observing 

describing reciting 

identifying selecting 

listing scanning 

Questions: 

"How does the picture make you feel?" Describing 
"The Mexican houses were made of 

mud bricks called what?" Completing 
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Making Sense Out of Information 
Gathering ^Processing) 



To cause the student to PROCESS the data gathered through the 
senses and retrieve from long- or short-term memory. 



Examples: 

synthesizing 

analyzing 

classifying 

comparing 

contrasting 

distinguishing 

experimenting 

Questions: 

"Compare the strength of steel to the 
strength of copper." 

"How can you arrange the rocks in the 
order of their size?" 

"How are pine needles different from 
redwood needles?" 



categorizing 

explaining 

grouping 

inferring 

making analogies 

organizing 

sequencing 



Comparing 
Sequencing 
Contrasting 



Raising Questions to Higher Levels 



1. 



Input 

How many 
of you are 
buying milk 
today? 



Processing 

Why are you buying 
milk today? Are there 
more students in Mr. 
Jones' room than in 
ours who are buying 
milk? Why are there 
fewer milk buyers 
in our room? 



Output 

What do you think 
would happen if 
nobody bought 
milk anymore? 
What do you think 
would happen if 
all the children 
in the world had 
all the milk they 
needed? Could you 
give some examples of 
countries where this is 
the case? 



33 

451 



2. What was the 
weather like 
yesterday? 



How does the weather 
today compare to the 
weather yesterday? 



What do you think the 
weather will be like 
tomorrow? 



Why is our weather so What can you say 
different today? about cities which 

have weather like 

ours? 

How does the weather 
in Washington, DC 
compare with the 
weather in Tokyo? 



A Model of Intellectual Functioning 



Intake of data Making sense out Applying and 



Input 



Processing 



Output 



through the 
senses 



of the data 



Evaluating 



Recalling from 
both short- and 
long-term 
memory 



Metacognition 
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Response to Dalton and Noir^s Presentation 

Victoria Jew 
California State University, Sacramento 

The focus of my discussion will be on Recommendation Four: that 
we design teacher preparation program evaluation tg examine pro- 
grammatic features that are important to the development of the 
candidates and that we examine a program's responsiveness to new 
knowledge. I think that we need to focus on two areas of this recom- 
mendation because of the nature of this particular field: one is 
training resiliency because of the school sites that our bilingual edu- 
cation candidate will eventually work in, the other is the responsive- 
ness to the changing demographics of the LEP students - this is in 
addition to the general responsiveness to the emergence of new 
knowledge. 

Let me focus on the first point: the condition under which our 
candidates ultimately have to teach. Bilingual education, as we all 
know, is still a controversial field. It is controversial because of the 
specific characteristics of the knowledge base, the population it 
serves, and the broader social and political climate under which the 
education of limited English proficient students operates. This con- 
dition still exists in a lot of the school settings in which our students 
will eventually work. 

Our graduates are often placed in non-nurturing or even hostile 
school environments because bilingual education and other ap- 
proaches to teaching limited English proficient students force schools 
to address language and culture issues and because they elevate the 
needs of non-mainstream students into an educational priority at 
these school sites. 

To add to this condition is the controversial nature of our knowl- 
edge base. Although we have a substantial knowledge base about 
language development and effective practices for LEP students, a 
substantial body of this knowledge is controversial because it may be 
contrary to the belief system of the school culture of the sites in 
which our graduates will have to work. Whereas in non-controver- 
sial areas of this knowledge base, training programs can merely ad- 
dress the program design in terms of effective implementation, such 
as: assisting the candidates in developing a professional perspective 
or working on their competency to bridge theory and practice. But 
in controversial areas, we really need to address additional program 
design features to look at the effect of training when it comes into 
contact with a hostile or non-nurturing environment. 
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J-^et me give you some examples about the nature of the body of 
knowledge that I am referring to. If you are looking at sheltered En- 
glish, whether you are looking at its theoretical and philosophical 
perspective or its approach and strategies, it is not controversial. So 
in training, it is something that can probably be dealt with by em- 
ploying regular measures for increasing the effectiveness of train- 
ing. I suspect language choice for instruction in a multilingual class- 
room will be just as non-controversial. But if you are to look at ap- 
proaches such as substantive and substantial use of the primary lan- 
guage for instruction, criteria for transition, or language mainte- 
nance, then you will run into areas where the specific school culture 
might be completely not in congruence with the knowledge base in 
which the new bilingual teacher has been trained. Another area of 
the knowledge base that was mentioned in the paper: I can more or 
less predict that the work Lily Wong Fillmore is doing right now that 
warns of the harmful effects of early English immersion for young 
children will be controversial when it gets to a school setting. 

Let me share with you one of my greatest frustration as a 
teacher trainer which is also the frustration many of us in teacher 
training share. We prepare bilingual teachers who appear to be well- 
trained as they complete the training program l. They appear to have 
at least a well defined philosophical perspective; they have developed 
a sound professional perspective; they show beginning level of com- 
petency in methodology, strategy, and practice that are effective in 
classrooms for LEP students. But within two years afler graduation, 
when we visit them in their teaching setting, a good number of these 
former students whom we have trained look no different from others 
at the site who have noL been trained. They have become socialized 
into this particular environment, and we see little sign of the train- 
ing they have received. 

For some of these students, the rhetoric of bilingual education 
remains, but there is no reflection of that particular perspective in 
their classroom practices. Others completely abandon what they 
have gained in the training program and look just like everyone else. 
Still others even work against their training, using the knowledge 
base they were trained in to attack bilingual education. I am won- 
dering, for those who have abandoned the skills that they developed 
when they were faced with a hostile environment, whether the de- 
sign of a training program can focus on preparing the candidates to 
face the special kind of challenge that bilingual educators often have 
to face: the lack of support, the lack of material, the lack of commit- 
ment, the lack of resources and the lack of status in schools. 

Some of the enhancement of the candidates can be addressed 
when you look at how we bridge theory and practice, how we assist 
students in practice teaching, how we coach students. But there are 
other areas that need something more than these types of improve- 
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ment in training. Therefore, I really think that data needs to be col- 
lected to explore ways in which the process of teacher development 
can incorporate the kind of training and support that would enhance 
the resiliency for the special kind of knowledge base that we work 
from; the belief system and the commitment to an effective education 
for language minority students with which a training program pre- 
pares, trains, and empowers bilingual educators. 

The second area I want to give focus to is the responsiveness to 
new knowledge. Within the consideration for new knowledge, we 
need to consider responsiveness to the changing demographics of the 
LEP student population in schools. We need to evaluate how effec- 
tively training programs continue to modify programs as they collect 
data about emerging populations of LEP students. Let me give you 
some examples of the kind of frustration that we have in the field of 
bilingual teacher training, particularly in the training of teachers for 
working with Asian-Pacific language populations. There is an abso- 
lute dearth of data or research on this particular population in terms 
of bilingual education or second language education in the U.S. This 
is also true for any language other than Spanish. In many ways, a 
lot of us, who are working in training programs that train for other 
languages, experience a tremendous amount of difficulties in pre- 
senting data that can be considered robust. 

In addition, there is a certain degree of what may be considered 
capriciousness in the way we determine what is generalizable or not 
generalizable to other LEP populations from the data we have about 
Spanish speaking students. Some of us are quite surprised by the 
type of interpretation that others in bilingual education render when 
it comes to non Spanish languages. One example, for instances, is 
the question of literacy transferability. In California, there has al- 
ways been a focus on the extent of transferability of literacy between 
LI and L2. This is all well and fine for two languages such as Span- 
ish and English, not very distant in many ways. Then when you 
have candidates in training who are from other language back- 
grounds, including those from languages with extremely different 
orthographies such as Chinese, the capricious manner in which one 
trainer determines that there is zero transferability and another de- 
termines to claim extensive transferability without any attention to 
the obvious differences can be most confusing. 

Another recent surprise is the Summary of the Longitudinal 
Study by Ramirez in which a statement was made that the results of 
the study cannot be generalized to other language groups, because 
research indicated that other language background students acquire 
English differently. There is no body of research that we are aware 
of about how other language groups acquire English. Given the lack 
of data for interpretation, this statement in the summary again is 
open to capricious interpretation that can be both surprising to 
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people who are speakers of these languages and confusing to candi- 
dates in training. 

Finally, we really need to start to take a look at how we can con- 
tinue to incorporate a pluralistic perspective in the way we train bi- 
lingual teachers. A lot of the paradigms for training or in classroom 
practices in bilingual education or in language development educa- 
tion have been set based on the experiences, practices, and data of 
the Spanish'Speaking population. Thus, our programs need to give 
focus to evaluation that examines how well we continue to include 
and how well we continue to change in order to address changing de- 
mographics. 
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